Multi-screen hold and drag gesture

ABSTRACT

Embodiments of a multi-screen hold and drag gesture are described. In various embodiments, a hold input is recognized at a first screen of a multi-screen system when the hold input is held in place. A motion input is recognized at a second screen of the multi-screen system, and the motion input is recognized to select a displayed object while the hold input remains held in place. A hold and drag gesture can then be determined from the recognized hold and motion inputs.

BACKGROUND

Computing devices, such as personal computers, laptop computers,tabletop computers, entertainment devices, and the like increasinglyoffer more functions and features which can make it difficult for a userto navigate and select application commands that are relevant to afunction the user wants to initiate on a device. The traditionaltechniques to interact with computing devices, such as a mouse,keyboard, and other input devices, may become less efficient as thefunctions and features of computing devices continues to increase. Achallenge that continues to face designers of these devices is how toincorporate interaction techniques that are not only intuitive, butallow a user to easily and quickly interact with the many functions andfeatures of a computing device.

SUMMARY

This summary is provided to introduce simplified concepts ofmulti-screen gestures. The simplified concepts are further describedbelow in the Detailed Description. This summary is not intended toidentify essential features of the claimed subject matter, nor is itintended for use in determining the scope of the claimed subject matter.

Embodiments of a multi-screen hold and drag gesture are described. Invarious embodiments, a hold input is recognized at a first screen of amulti-screen system when the hold input is held in place. A motion inputis recognized at a second screen of the multi-screen system, and themotion input is recognized to select a displayed object while the holdinput remains held in place. A hold and drag gesture can then bedetermined from the recognized hold and motion inputs. In otherembodiments, the hold and drag gesture can be used to maintain a displayof a first part of the displayed object on the first screen and drag asecond part of the displayed object that is displayed on the secondscreen to pocket the second part of the displayed object for asplit-screen view; maintain a display of a first part of the displayedobject on the first screen and drag a pocketed second part of thedisplayed object to expand the display on the second screen; maintain adisplay of the displayed object on the first screen and expand thedisplay of the displayed object onto the second screen; and/or initiatea multi-screen display of the displayed object.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of multi-screen gestures are described with reference to thefollowing drawings. The same numbers are used throughout the drawings toreference like features and components:

FIG. 1 illustrates an environment of a multi-screen system that canimplement various embodiments of multi-screen gestures.

FIG. 2 illustrates an example system with multiple devices that canimplement various embodiments of multi-screen gestures for a seamlessuser experience in ubiquitous environments.

FIG. 3 illustrates an example of multi-screen pinch and expand gestureson a multi-screen system.

FIG. 4 illustrates example method(s) for multi-screen pinch and expandgestures in accordance with one or more embodiments.

FIG. 5 illustrates an example of a multi-screen pinch-to-pocket gestureon a multi-screen system.

FIG. 6 illustrates example method(s) for a multi-screen pinch-to-pocketgesture in accordance with one or more embodiments.

FIG. 7 illustrates an example of a multi-screen dual tap gesture on amulti-screen system.

FIG. 8 illustrates example method(s) for a multi-screen dual tap gesturein accordance with one or more embodiments.

FIG. 9 illustrates an example of a multi-screen hold and tap gesture ona multi-screen system.

FIG. 10 illustrates example method(s) for a multi-screen hold and tapgesture in accordance with one or more embodiments.

FIG. 11 illustrates an example of a multi-screen hold and drag gestureon a multi-screen system.

FIG. 12 illustrates example method(s) for a multi-screen hold and draggesture in accordance with one or more embodiments.

FIG. 13 illustrates an example of a multi-screen hold and page-flipgesture on a multi-screen system.

FIG. 14 illustrates example method(s) for a multi-screen hold andpage-flip gesture in accordance with one or more embodiments.

FIG. 15 illustrates an example of a multi-screen bookmark hold gestureon a multi-screen system.

FIG. 16 illustrates example method(s) for a multi-screen bookmark holdgesture in accordance with one or more embodiments.

FIG. 17 illustrates an example of a multi-screen object-hold andpage-change gesture on a multi-screen system.

FIG. 18 illustrates example method(s) for a multi-screen object-hold andpage-change gesture in accordance with one or more embodiments.

FIG. 19 illustrates an example of a multi-screen synchronous slidegesture on a multi-screen system.

FIG. 20 illustrates example method(s) for a multi-screen synchronousslide gesture in accordance with one or more embodiments.

FIG. 21 illustrates components of an example device that can implementvarious embodiments of multi-screen gestures.

DETAILED DESCRIPTION

Embodiments of multi-screen gestures provide that a user of one or morecomputing devices in a multi-screen system can provide inputs at morethan one screen of the system to initiate computing device functions. Invarious embodiments of multi-screen gestures, a multi-screen systemincludes two or more screens that may be implemented as independentdevices, or integrated into a single, multi-screen device. A user caninput any type of various inputs or combinations of inputs, such asselect, hold, motion, touch, and/or tap inputs that are recognized atmultiple screens of a multi-screen system or multi-screen device. Amulti-screen gesture can then be recognized from a combination of thevarious inputs to initiate a computing device function. Accordingly, themulti-screen gestures provide that the user can provide the variousinputs to a multi-screen system or device in an intuitive manner, ratherthan by conventional techniques used to input commands to a computerdevice.

In various embodiments, multi-screen gestures can be implemented by acomputer device having multiple screens. Alternatively, multi-screengestures can be implemented by a multi-screen system of two or morescreens that may not be physically connected or integrated into a singledevice, but rather are communicatively linked such as via a data ornetwork connection. A multi-screen system can include multiple,independent slate or handheld devices that may automatically discoverone another, are explicitly paired by a user, or are otherwisepositioned in temporary physical proximity.

In various embodiments of multi-screen gestures, a multi-screen pinchgesture can be used to condense a displayed object on multiple screensof a multi-screen system or device. Alternatively, a multi-screen expandgesture can be used to expand a displayed object for display on multiplescreens of the multi-screen system or device. Multi-screen pinch andexpand gestures may also semantically zoom through different levels ofan information architecture associated with a display, object, and/orapplication. A multi-screen pinch-to-pocket gesture can be used topocket a displayed object, such as to save the displayed object as athumbnail image under a bezel of a multi-screen system or device.

A multi-screen dual tap gesture can be used to expand or pocket adisplayed object that is displayed on multiple screens of a multi-screensystem or device. For example, a displayed object can be expanded forfull-screen display on first and second screens when a dual tap gestureis determined while the displayed object is pocketed. Alternatively, adisplayed object can be pocketed when a dual tap gesture is determinedwhile the displayed object is displayed full-screen on the first andsecond screens.

A multi-screen hold and tap gesture can be used to move and/or copy adisplayed object from one displayed location to another, such as to moveor copy an object onto a journal page, or incorporate the object into anotebook. A multi-screen hold and drag gesture can be used to maintain adisplay of a first part of a displayed object on one screen and drag asecond part of the displayed object that is displayed on another screento pocket the second part of the displayed object for a split-screenview. Alternatively, a hold and drag gesture can be used to maintain adisplay of a first part of the displayed object on one screen and drag apocketed second part of the displayed object to expand the display onanother screen.

A multi-screen hold and page-flip gesture can be used to select ajournal page that is displayed on one screen and flip journal pages todisplay two additional or new journal pages, much like flipping pages ina book. The journal pages are flipped in a direction of the selectedjournal page to display the two new journal pages, much like flippingpages forward or backward in a book. Alternatively, a hold and page-flipgesture can be used to maintain the display of a journal page that isdisplayed on one screen and flip journal pages to display a differentjournal page on another screen. Non-consecutive journal pages can thenbe displayed side-by-side, which for a book, would involve tearing apage out of the book to place it in a non-consecutive page order to viewit side-by-side with another page.

A multi-screen bookmark hold gesture can be used to bookmark a journalpage at a location of a hold input to the journal page on a screen, andadditional journal pages can be flipped for viewing while the bookmarkis maintained for the journal page. A bookmark hold gesture mimics theaction of a reader holding a thumb or finger between pages to save aplace in a book while flipping through other pages of the book.Additionally, a bookmark is a selectable link back to the journal page,and a selection input of the bookmark flips back to display the journalpage on the screen. A multi-screen object-hold and page-change gesturecan be used to move and/or copy a displayed object from one displaylocation to another, such as to incorporate a displayed object fordisplay on a journal page. Additionally, a relative display position canbe maintained when a displayed object is moved or copied from onedisplay location to another.

A multi-screen synchronous slide gesture can be used to move a displayedobject from one screen for display on another screen, replace displayedobjects on the device screens with different displayed objects, movedisplayed objects to reveal a workspace on the device screens, and/orcycle through one or more workspaces (e.g., applications, interfaces,etc.) that are displayed on the system or device screens. A synchronousslide gesture may also be used to navigate to additional views, orreassign a current view to a different screen. Additionally, differentapplications or workspaces can be kept on a stack and cycled through,forward and back, with synchronous slide gestures.

While features and concepts of the described systems and methods for amulti-screen gestures can be implemented in any number of differentenvironments, systems, and/or various configurations, embodiments of amulti-screen gestures are described in the context of the followingexample systems and environments.

FIG. 1 illustrates an environment 100 in an example implementation thatis operable to employ multi-screen gesture techniques. The illustratedenvironment 100 includes an example of a computing device 102 that maybe configured in a variety of ways, such as any type of multi-screencomputer or device. For example, the computing device 102 may beconfigured as a computer (e.g., a laptop computer, notebook computer,tablet PC, tabletop computer, and so on), a mobile station, anentertainment appliance, a gaming device, and so forth as furtherdescribed with reference to FIG. 2. The computing device 102 may also beimplemented with software that causes the computing device 102 toperform one or more operations.

In this example environment 100, the computing device 102 is amulti-screen device that includes a first screen 104 and a second screen106, which can each be implemented as any type of display device,display system, and/or touch-screen. The first and second screens candisplay any type of background or desktop, as well as user interfacesand various displayable objects (e.g., any type of pictures, images,graphics, text, notes, sketches, drawings, selectable controls, userinterface elements, etc.). The first and second screens can also displayjournal pages, such as any type of notebook, periodical, book, paper, asingle page, and the like in an electronic form.

The computing device 102 includes a gesture module 108 that isrepresentative of functionality to determine gestures and causeoperations to be performed that correspond to the gestures. Thecomputing device also includes an input recognition system 110implemented to recognize various inputs or combinations of inputs, suchas a select input, hold input, motion input, touch input, tap input, andthe like. The input recognition system 110 may include any type of inputdetection features to distinguish the various types of inputs, such assensors, light sensing pixels, touch sensors, cameras, and/or a naturaluser interface that interprets user interactions, gestures, inputs, andmotions. In implementations, the input recognition system 110 can detectmotion inputs at the first or second screens from discernable variables,such as from a direction variable (e.g., right-to-left or vice-versa);from start region position variables (e.g., left1, top1, right1,bottom1) and end region position variables (e.g., left2, top2, right2,bottom2); and/or from a motion rate variable (e.g., a particular numberof pixels per second).

The input recognition system 110 recognizes the various types of inputs,and the gesture module 108 identifies or determines a multi-screengesture from the recognized inputs. For example, the input recognitionsystem 110 can recognize a first input at the first screen 104, such asa touch input 112, and recognize a second input at the second screen106, such as a select input 114. The gesture module 108 can thendetermine a type of multi-screen gesture from the recognized touch andselect inputs. An input at the first or second screen may also berecognized as including attributes (e.g., movement, a selection point,etc.) that differentiate one type of input from another input asrecognized by the input recognition system 110. This differentiation maythen serve as a basis to identify or determine a motion input from atouch input, and consequently an operation that is to be performed basedon a determination of the corresponding gesture. In implementations, thecomputing device 102 may include a gestures database that includesvarious determinable representations of gestures, inputs, and/or motionsfrom which the gesture module 108 can determine or identify amulti-screen gesture.

The computing device 102 can also be implemented to recognize anddifferentiate between the various inputs, such as a touch input and astylus input. The differentiation may be performed in a variety of ways,such as by recognizing the size of a finger input versus the size of astylus input. Differentiation may also be performed through use of acamera to distinguish a touch input (e.g., holding up one or morefingers), a stylus input (e.g., holding two fingers together to indicatea point), or an input via a natural user interface (NUI). A variety ofother techniques for distinguishing the various types of inputs arecontemplated.

A variety of different types of inputs may be recognized by the inputrecognition system 110, and a variety of different gestures may bedetermined by the gesture module 108, such a gestures that arerecognized as a single type of input as well as gestures involvingmultiple types of inputs. Accordingly, the gesture module 108 ofcomputing device 102 may include a bimodal input module 116 that isrepresentative of functionality to recognize inputs and identify ordetermine gestures involving bimodal inputs. The gesture module 108 maysupport a variety of different gesture techniques by recognizing andleveraging different types of inputs through use of the bimodal inputmodule 116. For instance, the bimodal input module 116 may be configuredto recognize a stylus as a writing tool, whereas touch is employed tomanipulate objects displayed on the first or second screen. It should benoted that by differentiating between the various types of inputs, thenumber of gestures that are made possible by each of these inputs aloneis also increased.

Accordingly, the gesture module 108 can support a variety of differentmulti-screen gestures 118, both bimodal and otherwise. Examples of themulti-screen gestures 118 described herein include pinch and expandgestures 120, a pinch-to-pocket gesture 122, a dual tap gesture 124, ahold and tap gesture 126, a hold and drag gesture 128, a hold andpage-flip gesture 130, a bookmark hold gesture 132, an object-hold andpage-change gesture 134, and a synchronous slide gesture 136. Each ofthese different multi-screen gestures is described in a correspondingsection in the following discussion. Although each multi-screen gestureis described in a different section, it should be readily apparent thatthe features of these gestures may be combined and/or separated tosupport additional gestures. Therefore, the description is not limitedto these examples. Additionally, although the following discussion maydescribe specific examples of select, hold, motion, touch, and tapinputs, the various types of inputs may be switched in differentinstances (e.g., a touch input may be used as a select input, and viceversa) and/or both inputs may be provided with the same input withoutdeparting from the spirit and scope thereof.

The illustrated environment 100 also includes an example of amulti-screen system 138 that includes two (or more) devices that eachhave a screen, such as a first device 140 that has a screen 142, and asecond device 144 that has a screen 146. The screens are not physicallyconnected or integrated into a single device, but rather arecommunicatively linked such as via a data or network connection. Amulti-screen system can include multiple, independent slate or handhelddevices that may automatically discover one another, are explicitlypaired by a user, or are otherwise positioned in temporary physicalproximity. In an implementation, a multi-screen system may also includea multi-screen device. The first device 140 and the second device 144 ofthe multi-screen system 138 can each be configured as described withreference to computing device 102 in any form of a computer (e.g., alaptop computer, notebook computer, tablet PC, tabletop computer, and soon), a mobile station, an entertainment appliance, a gaming device, andso forth.

FIG. 2 illustrates an example system 200 that includes the computingdevice 102 as described with reference to FIG. 1. The example system 200enables ubiquitous environments for a seamless user experience whenrunning applications on a personal computer (PC), a television device,and/or a mobile device. Services and applications run substantiallysimilar in all three environments for a common user experience whentransitioning from one device to the next while utilizing anapplication, playing a video game, watching a video, and so on.

In the example system 200, multiple devices are interconnected through acentral computing device. The central computing device may be local tothe multiple devices or may be located remotely from the multipledevices. In one embodiment, the central computing device is a “cloud”server farm, which comprises one or more server computers that areconnected to the multiple devices through a network, the Internet, orother data communication link. In one embodiment, this interconnectionarchitecture enables functionality to be delivered across multipledevices to provide a common and seamless experience to a user of themultiple devices. Each of the multiple devices may have differentphysical requirements and capabilities, and the central computing deviceuses a platform to enable the delivery of an experience to the devicethat is both tailored to the device and yet common to all devices. Inone embodiment, a class of target devices is created and experiences aretailored to the generic class of devices. A class of devices may bedefined by physical features, types of usage, or other commoncharacteristics of the devices.

In various implementations, the computing device 102 may assume avariety of different configurations, such as for computer 202, mobile204, and television 206 uses. Each of these configurations includesdevices that may have generally different constructs and capabilities,and thus the computing device 102 may be configured according to one ormore of the different device classes. For instance, the computing device102 may be implemented as the computer 202 class of a device thatincludes a personal computer, desktop computer, a multi-screen tabletopcomputer, laptop computer, netbook, and so on. The computing device 102may also be implemented as the mobile 204 class of device that includesmobile devices, such as a mobile phone, portable music player, portablegaming device, a tablet computer, a multi-screen tablet computer, and soon. The computing device 102 may also be implemented as the television206 class of device that includes devices having or connected togenerally larger screens in casual viewing environments. These devicesinclude televisions, set-top boxes, gaming consoles, and so on. Thetechniques described herein may be supported by these variousconfigurations of the computing device 102 and are not limited to thespecific examples of multi-screen gestures described in the followingsections.

The cloud 208 includes and/or is representative of a platform 210 forserver-based services 212. The platform 210 abstracts underlyingfunctionality of hardware (e.g., servers) and software resources of thecloud 208. The server-based services 212 may include applications and/ordata that can be utilized while all or most of the computer processingis executed on servers that are remote from the computing device 102.Server-based services can be provided as a service over the Internetand/or through a subscriber network, such as a cellular or WiFi network.

The platform 210 may abstract resources and functions to connect thecomputing device 102 with other computing devices. The platform 210 mayalso serve to abstract scaling of resources to provide a correspondinglevel of scale to encountered demand for the server-based services 212that are implemented via the platform 210. Accordingly, in aninterconnected device embodiment, implementation of functionality of thegesture module 108 may be distributed throughout the system 200. Forexample, the gesture module 108 may be implemented in part on thecomputing device 102 as well as via the platform 210 that abstracts thefunctionality of the cloud 208.

Further, the functionality may be supported by the computing device 102in any one or more of the configurations. For example, the multi-screengesture techniques supported by the gesture module 108 and the inputrecognition system 110 may be recognized using track pad functionalityin the computer 202 configuration, touch-screen functionality in themobile 204 configuration, and/or recognized by a camera as part of anatural user interface (NUI) that does not involve contact with aspecific input device in the television 206 configuration. Further,performance of the operations to detect and recognize the inputs toidentify or determine a particular multi-screen gesture may bedistributed throughout the system 200, such as by the computing device102 and/or the server-based services 212 supported by the platform 210of the cloud 208.

In addition to the following sections that describe the variousmulti-screen gestures, example methods are also described with referenceto respective figures in accordance with various embodiments ofmulti-screen gestures. Generally, any of the functions, methods,procedures, components, and modules described herein can be implementedusing software, firmware, hardware (e.g., fixed logic circuitry), manualprocessing, or any combination thereof. A software implementationrepresents program code that performs specified tasks when executed by acomputer processor. The example methods may be described in the generalcontext of computer-executable instructions, which can include software,applications, routines, programs, objects, components, data structures,procedures, modules, functions, and the like. The program code can bestored in one or more computer-readable memory devices, both localand/or remote to a computer processor. The methods may also be practicedin a distributed computing environment by multiple computer devices.Further, the features described herein are platform-independent and canbe implemented on a variety of computing platforms having a variety ofprocessors.

Multi-Screen Pinch and Expand Gestures

FIG. 3 illustrates examples 300 of multi-screen pinch and expandgestures on a multi-screen system 302, which in these examples, is shownas a two-screen device. The multi-screen system 302 may be implementedas any of the various devices described with reference to FIGS. 1 and 2.In this example, the multi-screen system 302 includes a first screen 304and a second screen 306, each implemented to display any type of userinterface and various displayable objects (e.g., any type of pictures,images, graphics, text, notes, sketches, drawings, selectable controls,user interface elements, etc.). The screens can also display journalpages, such as any type of notebook, periodical, book, paper, singlepage, and the like in an electronic form. The multi-screen system 302can include a gesture module 108 and an input recognition system 110, asdescribed with reference to the computing device 102 shown in FIG. 1,and may also be implemented with any combination of components asdescribed with reference to the example device shown in FIG. 21.Although the examples are illustrated and described with reference tothe two-screen device, embodiments of multi-screen pinch and expandgestures can be implemented by a multi-screen system having more thantwo screens.

A multi-screen pinch gesture can be used to condense a displayed objecton multiple screens of a multi-screen system. Alternatively, amulti-screen expand gesture can be used to expand a displayed object fordisplay on multiple screens of the multi-screen system. In the firstview 308 of the multi-screen system 302, a first journal page 310 isdisplayed on the first screen 304, and a second journal page 312 isdisplayed on the second screen 306. The input recognition system 110 isimplemented to recognize a first input 314 at the first screen 304,where the first input also includes a first motion input 316. The inputrecognition system 110 can also recognize a second input 318 at thesecond screen 306, where the second input also includes a second motioninput 320, and the second input is recognized approximately when thefirst input is recognized.

The gesture module 108 is implemented to determine the multi-screenpinch gesture from the motion inputs 316, 320 that are associated withthe recognized first and second inputs 314, 318. The pinch gesture canbe identified as a cross-screen combination of the first and secondmotion inputs that are effective to condense the displayed journal pages310, 312. In an implementation, the input recognition system 110 canrecognize that a distance between the first and second inputs changes(e.g., decreases) with the motion inputs. The change in distance mayalso have minimum distance thresholds. The gesture module 108 can thendetermine the pinch gesture from a decrease in the distance between thefirst and second inputs.

In some embodiments, a multi-screen pinch gesture is determined when thegesture motion inputs are recognized within a defined zone or regionproximate the edge shared by the screens, such as a bezel that separatesthe first and second screens on the multi-screen device. The zone orregion proximate the bezel can be defined as a minimum distance from theedge or bounding rectangle where a pinch gesture is recognized. In otherembodiments, segments of a pinch gesture may be incrementallyrecognized, such as when a pinch gesture is composed of: approximatesynchronous inputs (e.g., finger-touch-contacts) on adjacent edges; thefirst input 314 holds while the second motion input 320 slides towardthe bezel (e.g., one finger holding while the other finger slides towardthe common edge); or approximate synchronous lifts of both fingersresulting in a composite pinch gesture. Additionally, a user can inputexpand and pinch gestures in opposite directions back and forth betweenthe gesture states until lifting the first and second inputs. Similar toa double-tap gesture on a user interface, applications can subscribe toa composite high-level pinch and/or expand gesture that includes some orall of the gesture segments.

The second view 322 of the multi-screen system 302 illustrates atransition of the journal pages 310, 312 being condensed from anoriginal position 324 in a direction 326 in response to the pinchgesture. The third view 328 of the multi-screen system 302 illustratesthe journal pages 310, 312 condensed for display. The pinch gesturegives the appearance of zooming-out when a displayed object iscondensed. In this example, the pinch gesture condenses the journalpages, zooming-out to a virtual desktop 330 on the multi-screen system302. The virtual desktop 330 may be used as a space to navigate to otherjournals or books, drag displayed objects between journal pages, orleave reminders such as posted notes and to-do lists that are visiblefor quick access outside of any particular individual notebook, e-book,journal, or document. Alternate navigable views may include: anorganizational view of thumbnail images of multiple pages of a notebook(e.g., a “light table view”); a minimized or shrunken-down version ofthe current notebook with multiple pages, page tabs, and/or bookmarksprotruding from the notebook, and a surround similar to the virtualdesktop 330 (e.g., a “butterfly view”); a “library view” across multiplebooks and/or journals; or a home screen.

From the third view 328, a multi-screen expand gesture can be used toreturn to the full-screen view of the journal pages, such as shown inthe first view 308. The gesture module 108 is also implemented todetermine the multi-screen expand gesture that can be identified as across-screen combination of motion inputs that are effective to expandthe journal pages 310, 312 from the condensed display shown in the thirdview 328 of the multi-screen system. In an implementation, the inputrecognition system 110 can recognize that a distance between inputschanges (e.g., increases) with motion inputs. The gesture module 108 canthen determine the expand gesture from an increase in the distancebetween inputs. A transition from the third view 328 back to the firstview 308 of the multi-screen system 302 illustrates that the journalpages 310, 312 are expanded for full-screen display on the first andsecond screens. The expand gesture gives the appearance of zooming-inwhen a displayed object is expanded.

It should be noted that the representations of the first and secondinputs, as well as the indications of motion directions, are merelyillustrative for discussion purposes and may or may not appear on thescreens of the multi-screen system when described embodiments areimplemented. Additionally, any description herein of an input or motionat one screen that may correlate to another input or motion at anotherscreen is applicable to either the first or second screens of themulti-screen system. Additionally, a three-, four-, or five-fingermulti-screen pinch or expand gesture that spans two or more screens isalso contemplated, as are two-handed stretch and squeeze gestures thatmay be recognized and determined from multiple finger and/or contactinputs.

FIG. 4 illustrates example method(s) 400 of multi-screen pinch andexpand gestures. The order in which the method is described is notintended to be construed as a limitation, and any number of thedescribed method blocks can be combined in any order to implement themethod, or an alternate method.

At block 402, a first input is recognized at a first screen of amulti-screen system, the first input including a first motion input. Forexample, the input recognition system 110 recognizes the first input 314at the first screen 304 of the multi-screen system 302, the first inputincluding the first motion input 316. At block 404, a second input isrecognized at a second screen of the multi-screen system, the secondinput including a second motion input. For example, the inputrecognition system 110 also recognizes the second input 318 at thesecond screen 306, the second input including the second motion input320, and the second input being recognized approximately when the firstinput is recognized. Alternatively or in addition, the first input 314at the first screen 304 may initiate a time-out (e.g., 500 ms) with theinput recognition system 110 after which the first input is processedfor other single screen gestures if the second input is not provided.

At block 406, a change in distance between the first and second inputsis recognized based on the first and second motion inputs. For example,the input recognition system 110 recognizes that a distance between thefirst and second inputs 314, 318 changes (e.g., increases or decreases)with the motion inputs. At block 408, a determination is made as towhether the distance change between the first and second inputs is adecrease in the distance.

If the distance decreases between the first and second inputs (i.e.,“yes” from block 408), then at block 410, a pinch gesture is determined,which gives the appearance of zooming-out when a displayed object iscondensed. For example, the gesture module 108 determines the pinchgesture based on the first and second motion inputs that decrease thedistance between the first and second inputs. The pinch gesture can beidentified as a cross-screen combination of the first and second motioninputs that are effective to condense a displayed object, such as thedisplayed journal pages 310, 312. The pinch gesture gives the appearanceof zooming-out when a displayed object is condensed.

If the distance increases between the first and second inputs (i.e.,“no” from block 408), then at block 412, an expand gesture isdetermined, which gives the appearance of zooming-in when a displayedobject is expanded. For example, the gesture module 108 determines theexpand gesture based on the first and second motion inputs that increasethe distance between the first and second inputs. The expand gesture canbe identified as a cross-screen combination of the first and secondmotion inputs that are effective to expand a displayed object, such aswhen the displayed journal pages 310, 312 are expanded for full-screendisplay on the first and second screens of the multi-screen system 302.

Multi-Screen Pinch-to-Pocket Gesture

FIG. 5 illustrates examples 500 of a multi-screen pinch-to-pocketgesture on a multi-screen system 502, which in these examples, is shownas a two-screen device. The multi-screen system 502 may be implementedas any of the various devices described with reference to FIGS. 1 and 2.In this example, the multi-screen system 502 includes a first screen 504and a second screen 506, each implemented to display any type of userinterface and various displayable objects (e.g., any type of pictures,images, graphics, text, notes, sketches, drawings, selectable controls,user interface elements, etc.). The screens can also display journalpages, such as any type of notebook, periodical, book, paper, singlepage, and the like in an electronic form. The multi-screen system 502can include a gesture module 108 and an input recognition system 110, asdescribed with reference to the computing device 102 shown in FIG. 1,and may also be implemented with any combination of components asdescribed with reference to the example device shown in FIG. 21.Although the examples are illustrated and described with reference tothe two-screen device, embodiments of a multi-screen pinch-to-pocketgesture can be implemented by a multi-screen system having more than twoscreens.

A multi-screen pinch-to-pocket gesture can be used to pocket a displayedobject, such as to save the displayed object as a thumbnail image undera bezel of a multi-screen system. In the first view 508 of themulti-screen system 502, a first journal page 510 is displayed on thefirst screen 504, and a second journal page 512 is displayed on thesecond screen 506. The input recognition system 110 is implemented torecognize a first motion input 514 to a first screen region 516 at thefirst screen 504, where the first motion input is recognized when thefirst journal page 510 is selected. The input recognition system 110 canalso recognize a second motion input 518 to a second screen region 520at the second screen 506, where the second motion input is recognizedwhen the second journal page 512 is selected. The first screen region516 of the first screen 504 and the second screen region 520 of thesecond screen 504 are shown in a second view 522 of the multi-screensystem 502.

The gesture module 108 is implemented to determine the pinch-to-pocketgesture from the recognized motion inputs 514, 518. The pinch-to-pocketgesture can be identified as a cross-screen combination of the first andsecond motion inputs that are effective to condense the displayedjournal pages 510, 512 within the first and second screen regions 516,520 and pocket the journal pages proximate a bezel 524 that separatesthe first and second screens. Optionally, the gesture module 108 mayalso determine the pinch-to-pocket gesture from the first and secondmotion inputs 514, 518 that decrease a distance between a first input tothe journal page 510 on the first screen 504 and a second input to thesecond journal page 512 on the second screen 506.

The second view 522 of the multi-screen system 502 illustrates atransition of the journal pages 510, 512 being condensed from anoriginal position 526 in a direction 528 in response to thepinch-to-pocket gesture. The third view 530 of the multi-screen system502 illustrates the displayed object (e.g., the journal pages) pocketedproximate the bezel 524 and saved as a thumbnail image 532 for display.In this example, more of the virtual desktop 534 is displayed, and anyof the other displayed objects on the desktop are accessible while thejournal pages are pocketed as the thumbnail image 532. In anotherexample, a displayed object 536 (e.g., shown displayed as a sketch ofthe text “zeal” on the computing device 102 in FIG. 1) is pocketed underthe bezel 524 of the multi-screen system 502.

When displayed objects are pocketed to display the virtual desktop 534for access to the many other displayed objects, a user can interleavemultiple tasks in multiple journals or application views, and theneasily return to the pocketed items. Additionally, a pocketed item canbe dropped onto a notebook or journal page of an open notebook toincorporate the item in the context of other work and notations.

In various embodiments, a multi-screen pinch-to-pocket gesture may beused as a general mechanism for multi-tasking between different workingsets of screen views and/or applications. For example, if a Web browseris displayed on the first screen 504 and a journal page is displayed onthe second screen 506, then a user can pinch-to-pocket that pairing ofscreen views. A user may also pinch-to-pocket multiple screen views, inwhich case the set of pocketed views along the bezel 524 of the deviceappear as a taskbar from which the user can alternate between differentapplications and views.

In embodiments, the thumbnail image 532 of the journal pages is saved toa visual clip-board when pocketed. Additionally, the thumbnail image 532may be displayed on the first and/or second screens as a selectable linkto the journal pages when the displayed object is pocketed. From thisthird view 530, the input recognition system 110 can recognize a selectinput that the gesture module 108 determines as a tap gesture on thethumbnail image 532 that is effective to expand the journal pages 510,512 for display on the first and second screens, such as shown in thefirst view 508 of the multi-screen system 502.

It should be noted that the representations of the first and secondinputs and the indications of the motion directions, as well as thescreen regions, are merely illustrative for discussion purposes and mayor may not appear on the screens of the multi-screen system whendescribed embodiments are implemented. Additionally, any descriptionherein of an input or motion at one screen that may correlate to anotherinput or motion at another screen is applicable to either the first orsecond screens of the multi-screen system.

FIG. 6 illustrates example method(s) 600 of a multi-screenpinch-to-pocket gesture. The order in which the method is described isnot intended to be construed as a limitation, and any number of thedescribed method blocks can be combined in any order to implement themethod, or an alternate method.

At block 602, a first motion input to a first screen region isrecognized at a first screen of a multi-screen system, the first motioninput being recognized to select a displayed object. For example, theinput recognition system 110 recognizes the first motion input 514 tothe first screen region 516 at the first screen 504, and the firstmotion input is recognized when the first journal page 510 is selected.At block 604, a second motion input to a second screen region isrecognized at a second screen of the multi-screen system, the secondmotion input being recognized to select the displayed object. Forexample, the input recognition system 110 also recognizes the secondmotion input 518 to the second screen region 520 at the second screen506, and the second motion input is recognized when the second journalpage 512 is selected.

At block 606, a pinch-to-pocket gesture is determined from therecognized first and second motion inputs within the respective firstand second screen regions. For example, the gesture module 108determines the pinch-to-pocket gesture from the recognized motion inputs514, 518. The pinch-to-pocket gesture can be identified as across-screen combination of the first and second motion inputs that areeffective to condense the displayed journal pages 510, 512 within thefirst and second screen regions 516, 520 and pocket the journal pagesproximate the bezel 524 that separates the first and second screens.Alternatively or in addition, the pinch-to-pocket gesture is determinedfrom the first and second motion inputs that decrease a distance betweena first input to the first journal page 510 on the first screen and asecond input to the second journal page 512 on the second screen.

At block 608, the displayed object is pocketed proximate a bezel of themulti-screen system that separates the first and second screens. Forexample, the journal pages 510, 512 (e.g., the displayed object) arepocketed proximate the bezel 524 and saved as a thumbnail image 532 fordisplay. In an embodiment, the thumbnail image 532 is a selectable linkto the journal pages pocketed and/or the displayed object is saved to avisual clip-board.

At block 610, a select input is recognized as a tap gesture on thedisplayed object that is pocketed and, at block 612, the displayedobject is expanded for display on the first and second screens inresponse to the tap gesture. For example, the input recognition system110 recognizes a select input that the gesture module 108 determines asa tap gesture on the thumbnail image 532 that is effective to expand thejournal pages 510, 512 for display on the first and second screens ofthe multi-screen system 502.

Multi-Screen Dual Tap Gesture

FIG. 7 illustrates examples 700 of a multi-screen dual tap gesture on amulti-screen system 702, which in these examples, is shown as atwo-screen device. The multi-screen system 702 may be implemented as anyof the various devices described with reference to FIGS. 1 and 2. Inthis example, the multi-screen system 702 includes a first screen 704and a second screen 706, each implemented to display any type of userinterface and various displayable objects (e.g., any type of pictures,images, graphics, text, notes, sketches, drawings, selectable controls,user interface elements, etc.). The screens can also display journalpages, such as any type of notebook, periodical, book, paper, singlepage, and the like in an electronic form. The multi-screen system 702can include a gesture module 108 and an input recognition system 110, asdescribed with reference to the computing device 102 shown in FIG. 1,and may also be implemented with any combination of components asdescribed with reference to the example device shown in FIG. 21.Although the examples are illustrated and described with reference tothe two-screen device, embodiments of a multi-screen dual tap gesturecan be implemented by a multi-screen system having more than twoscreens.

A multi-screen dual tap gesture can be used to expand or pocket adisplayed object that is displayed on multiple screens of a multi-screensystem. For example, a displayed object can be expanded for full-screendisplay on first and second screens when a dual tap gesture isdetermined while the displayed object is pocketed. Alternatively, adisplayed object can be pocketed when a dual tap gesture is determinedwhile the displayed object is displayed full-screen on the first andsecond screens.

In the first view 708 of the multi-screen system 702, a first journalpage 710 is displayed on the first screen 704, and a second journal page712 is displayed on the second screen 706. The input recognition system110 is implemented to recognize a first tap input 714 to the firstjournal page 710 at the first screen 704. The input recognition system110 can also recognize a second tap input 716 to the second journal page712 at the second screen 706, where the second tap input is recognizedapproximately when the first tap input is recognized.

Alternatively, a single input (e.g., with a finger, thumb, palm, etc.)may contact both the first and second screens at approximately the sametime to initiate a dual-tap gesture input. For example, the multi-screendevice may have little or no spine, housing, or bezel between thescreens, in which case a single input can contact both screens together.Further, a multi-screen system with two (or more) independent screensmay be positioned such that a thumb or finger between the screens (e.g.,like a finger placed between pages in a book) makes contact with bothscreens.

The gesture module 108 is implemented to determine the multi-screen dualtap gesture from the recognized tap inputs 714, 716. The dual tapgesture can be identified as a cross-screen combination of the first andsecond tap inputs. The second view 718 of the multi-screen system 702illustrates that the dual tap gesture is effective to pocket the journalpages as a thumbnail image 720 proximate a bezel 722 of the multi-screensystem that separates the first and second screens. In this example, thevirtual desktop 724 is displayed, and any of the other displayed objectson the desktop are accessible while the journal pages are pocketed asthe thumbnail image 720.

The second view 718 of the multi-screen system 702 also illustrates thata dual tap gesture is effective to expand a displayed object for displayon the first and second screens of the multi-screen system. For example,the input recognition system 110 is implemented to recognize a first tapinput 726 to the thumbnail image 720 at the first screen 704, andrecognize a second tap input 728 to the thumbnail image 720 at thesecond screen 706, where the second tap input is recognizedapproximately when the first tap input is recognized. The gesture module108 can then determine the multi-screen dual tap gesture from therecognized tap inputs 726, 728, and the dual tap gesture effective toexpand the journal pages 710, 712 for display on the first and secondscreens as shown in the first view 708 of the multi-screen system 702.

The third view 730 of the multi-screen system 702 illustrates asplit-screen view that includes a first part of the displayed objectbeing displayed full-screen on the first screen and a second part of thedisplayed object being displayed condensed on the second screen. Forexample, the first journal page 710 is displayed full-screen on thefirst screen 704, and the second journal page 712 is pocketed fordisplay on the second screen 706. In an implementation, the inputrecognition system 110 can recognize a single select input to one of thejournal pages 710, 712 on one of the first or second screens, such asone of the tap inputs 726, 728 shown in the second view 718 of themulti-screen system 702. The single tap input is effective to initiatethe split-screen view of the journal pages, such as shown in the thirdview 730 of the multi-screen system 702.

It should be noted that the representations of the first and secondinputs are merely illustrative for discussion purposes and may or maynot appear on the screens of the multi-screen system when describedembodiments are implemented. Additionally, any description herein of aninput or motion at one screen that may correlate to another input ormotion at another screen is applicable to either the first or secondscreens of the multi-screen system.

FIG. 8 illustrates example method(s) 800 of a multi-screen dual tapgesture. The order in which the method is described is not intended tobe construed as a limitation, and any number of the described methodblocks can be combined in any order to implement the method, or analternate method.

At block 802, a first tap input to a displayed object is recognized at afirst screen of a multi-screen system. For example, the inputrecognition system 110 recognizes the first tap input 714 to the firstjournal page 710 at the first screen 704. At block 804, a second tapinput to the displayed object is recognized at a second screen of themulti-screen system, the second tap input being recognized approximatelywhen the first tap input is recognized. For example, the inputrecognition system 110 also recognizes the second tap input 716 to thesecond journal page 712 at the second screen 706, and the second tapinput is recognized approximately when the first tap input isrecognized.

At block 806, a dual tap gesture is determined from the recognized firstand second tap inputs. For example, the gesture module 108 determinesthe multi-screen dual tap gesture from the recognized tap inputs 714,716. The dual tap gesture can be identified as a cross-screencombination of the first and second tap inputs, and the dual tap gestureis effective to expand or pocket a displayed object that is displayed onthe first and second screens of the multi-screen system 702.Alternatively, a single input (e.g., with a finger, thumb, palm, etc.)that contacts both the first and second screens at approximately thesame time may be recognized and determined as a dual-tap gesture input.In embodiments, a displayed object can be expanded for full-screendisplay on the first and second screens when a dual tap gesture isdetermined while the displayed object is pocketed. Alternatively, adisplayed object can be pocketed when a dual tap gesture is determinedwhile the displayed object is displayed full-screen on the first andsecond screens.

At block 808, a single select input to the displayed object isrecognized on one of the first or second screens effective to initiate asplit-screen view of the displayed object. For example, the inputrecognition system 110 recognizes a single select input to one of thejournal pages 710, 712 on one of the first or second screens, such asone of the tap inputs 726, 728 shown in the second view 718 of themulti-screen system 702. The single tap input is effective to initiatethe split-screen view of the journal pages, such as shown in the thirdview 730 of the multi-screen system 702.

Multi-Screen Hold and Tap Gesture

FIG. 9 illustrates examples 900 of a multi-screen hold and tap gestureon a multi-screen system 902, which in these examples, is shown as atwo-screen device. The multi-screen system 902 may be implemented as anyof the various devices described with reference to FIGS. 1 and 2. Inthis example, the multi-screen system 902 includes a first screen 904and a second screen 906, each implemented to display any type of userinterface and various displayable objects (e.g., any type of pictures,images, graphics, text, notes, sketches, drawings, selectable controls,user interface elements, etc.). The screens can also display journalpages, such as any type of notebook, periodical, book, paper, singlepage, and the like in an electronic form. The multi-screen system 902can include a gesture module 108 and an input recognition system 110, asdescribed with reference to the computing device 102 shown in FIG. 1,and may also be implemented with any combination of components asdescribed with reference to the example device shown in FIG. 21.Although the examples are illustrated and described with reference tothe two-screen device, embodiments of a multi-screen hold and tapgesture can be implemented by a multi-screen system having more than twoscreens.

A multi-screen hold and tap gesture can be used to move and/or copy adisplayed object from one displayed location to another, such as to moveor copy an object onto a journal page, or incorporate the object into anotebook. In various embodiments, general functionality may include: ahold input to a command on one screen and a tap input on the otherscreen to apply the command on the other screen; a hold input to aparameter value (e.g., a color, brush thickness, image effect, filter,and the like) and a tap input on the other screen to apply the parametervalue to an object displayed on the other screen; and/or a hold input toa tag, category, or other metadata, and a tap input to apply the featureto an object displayed on the other screen. In one example, a journal ornotebook may include custom stickers that may be viewed on a page of thejournal or notebook. A sticker can be held on one page (e.g., asdisplayed on one screen) and then tapped to apply the sticker at the taplocation on another page (e.g., on the other screen). The stickers mayhave specific semantics attached to them, such as “Expense”, “To-do”,“Personal”, “Recipe”, or the like, and a sticker can be utilized to tagcontent to facilitate subsequent search and organization.

In the first view 908 of the multi-screen system 902, a journal page 910is displayed on the first screen 904, and various objects, such asdisplayed object 912, are displayed on the virtual desktop 914 on thesecond screen 906. The input recognition system 110 is implemented torecognize a hold input 916 at the second screen 906, where the holdinput is recognized when held to select the displayed object 912 on thesecond screen 906. The input recognition system 110 can also recognize atap input 918 at the first screen 904, where the tap input is recognizedwhile the displayed object 912 is selected on the second screen 906.

The gesture module 108 is implemented to determine the multi-screen holdand tap gesture from the recognized hold and tap inputs 916, 918. Thehold and tap gesture can be identified as a cross-screen combination ofthe hold and tap inputs, and the gesture is effective to move thedisplayed object 912 from a displayed location on the second screen 906to a tap input location for display on the first screen 904, asindicated at 920. The second view 922 of the multi-screen system 902illustrates that the dual tap gesture is effective to move the displayedobject 912 from the displayed location 924 on the second screen 906 andincorporate the displayed object 912 for display on the journal page 910that is displayed at the tap input location 926 on the first screen 904.The third view 928 of the multi-screen system 902 illustrates that thedual tap gesture is effective to copy the displayed object 912 togenerate an object copy 930, and initiate a display of the object copy930 at a tap input location 932 on the first screen 904.

In other embodiments of a multi-screen hold and tap gesture, the inputrecognition system 110 can recognize the tap input 918 to an additionaldisplayed object (e.g., the journal page 910) on the first screen 904,and the hold and tap gesture is then effective to correlate thedisplayed object 912 with the additional displayed object (e.g.,correlate the displayed object 912 with the journal page 910).Additionally, a displayed object may represent a function, and the holdand tap gesture is effective to apply the function of the displayedobject to an additional displayed object at a tap input location onfirst or second screen of the multi-screen system 902.

It should be noted that the representations of the hold and tap inputsare merely illustrative for discussion purposes and may or may notappear on the screens of the multi-screen system when describedembodiments are implemented. Additionally, any description herein of aninput or motion at one screen that may correlate to another input ormotion at another screen is applicable to either the first or secondscreens of the multi-screen system.

FIG. 10 illustrates example method(s) 1000 of a multi-screen hold andtap gesture. The order in which the method is described is not intendedto be construed as a limitation, and any number of the described methodblocks can be combined in any order to implement the method, or analternate method.

At block 1002, a hold input is recognized at a first screen of amulti-screen system, the hold input being recognized when held to selecta displayed object on the first screen. For example, the inputrecognition system 110 recognizes the hold input 916 at screen 906, andthe hold input is recognized when held to select the displayed object912 on screen 906. At block 1004, a tap input is recognized at a secondscreen of the multi-screen system, the tap input being recognized whilethe displayed object is selected. For example, the input recognitionsystem 110 also recognizes the tap input 918 at screen 904, and the tapinput is recognized while the displayed object 912 is selected on screen906. In an embodiment, the tap input may be recognized as a tap input toan additional displayed object on the second screen, and the hold andtap gesture is effective to correlate the displayed object with theadditional displayed object.

At block 1006, a hold and tap gesture is determined from the recognizedhold and tap inputs. For example, the gesture module 108 determines themulti-screen hold and tap gesture from the recognized hold and tapinputs 916, 918, and the hold and tap gesture can be identified as across-screen combination of the hold and tap inputs. In variousembodiments, the hold and tap gesture is effective to move the displayedobject from a displayed location on the first screen to a tap inputlocation for display on the second screen (at block 1008); incorporatethe displayed object for display on a journal page that is displayed ata tap input location on the second screen (at block 1010); copy thedisplayed object to generate an object copy, and display the object copyat a tap input location on the second screen (at block 1012); and/orapply a function of the displayed object to an additional displayedobject at a tap input location on the second screen (at block 1014).

Multi-Screen Hold and Drag Gesture

FIG. 11 illustrates examples 1100 of a multi-screen hold and draggesture on a multi-screen system 1102, which in these examples, is shownas a two-screen device. The multi-screen system 1102 may be implementedas any of the various devices described with reference to FIGS. 1 and 2.In this example, the multi-screen system 1102 includes a first screen1104 and a second screen 1106, each implemented to display any type ofuser interface and various displayable objects (e.g., any type ofpictures, images, graphics, text, notes, sketches, drawings, selectablecontrols, user interface elements, etc.). The screens can also displayjournal pages, such as any type of notebook, periodical, book, paper,single page, and the like in an electronic form. The multi-screen system1102 can include a gesture module 108 and an input recognition system110, as described with reference to the computing device 102 shown inFIG. 1, and may also be implemented with any combination of componentsas described with reference to the example device shown in FIG. 21.Although the examples are illustrated and described with reference tothe two-screen device, embodiments of a multi-screen hold and draggesture can be implemented by a multi-screen system having more than twoscreens.

A multi-screen hold and drag gesture can be used to maintain a displayof a first part of a displayed object on one screen and drag a secondpart of the displayed object that is displayed on another screen topocket the second part of the displayed object for a split-screen view.Alternatively, a hold and drag gesture can be used to maintain a displayof a first part of the displayed object on one screen and drag apocketed second part of the displayed object to expand the display onanother screen. The direction of a dragging gesture may also bedetermined based on different semantics (e.g., motion up, down, towardsthe bezel, away from the bezel, etc.). For the multi-screen hold anddrag gesture, four to eight cardinal directions can be defined fordifferent actions.

In the first view 1108 of the multi-screen system 1102, a first journalpage 1110 is displayed on the first screen 1104, and a second journalpage 1112 is displayed on the second screen 1106. The input recognitionsystem 110 is implemented to recognize a hold input 1114 at the firstscreen 1104, where the hold input is recognized when held in place. Theinput recognition system 110 can also recognize a motion input 1116 atthe second screen 1106, where the motion input is recognized to select adisplayed object (e.g., the journal page 1112) while the hold inputremains in place.

The gesture module 108 is implemented to determine the multi-screen holdand drag gesture from the recognized hold and motion inputs 1114, 1116.The hold and drag gesture can be identified as a cross-screencombination of the hold and motion inputs, and the gesture is effectiveto maintain the display of the first journal page 1110 on the firstscreen 1104 and drag the second journal page 1112 that is displayed onthe second screen 1106 to pocket the second journal page for asplit-screen view of the journal pages. The second view 1118 of themulti-screen system 1102 illustrates that the first journal page 1110 ismaintained for display on the first screen 1104, and the second journalpage 1112 is pocketed proximate the bezel 1120 of the multi-screensystem on the second screen 1106 for a split-screen view of the journalpages in response to the hold and drag gesture. In an embodiment, thesecond journal page 1112 is pocketed as a thumbnail image, which mayalso be a selectable link to the second journal page 1112.

The third view 1122 of the multi-screen system 1102 illustrates amulti-screen hold and drag gesture can be used to maintain a display ofa first part of a displayed object on one screen and drag a pocketedsecond part of the displayed object to expand the display on anotherscreen, or initiate a multi-screen display of the displayed object. Forexample, the input recognition system 110 can recognize a hold input1124 at the first screen 1104, where the hold input is recognized whenheld in place. The input recognition system 110 can also recognize amotion input 1126 at the second screen 1106, where the motion input isrecognized to select the second journal page 1112 when the journal pageis pocketed (e.g., the journal page 1112 as shown in the second view1118) while the hold input remains in place (e.g., to hold the firstjournal page 1110). The gesture module 108 can determine a multi-screenhold and drag gesture from the recognized hold and motion inputs 1124,1126, effective to expand the pocketed second journal page 1112 in adirection 1128 for display on the second screen 1106.

It should be noted that the representations of the hold and motioninputs are merely illustrative for discussion purposes and may or maynot appear on the screens of the multi-screen system when describedembodiments are implemented. Additionally, any description herein of aninput or motion at one screen that may correlate to another input ormotion at another screen is applicable to either the first or secondscreens of the multi-screen system.

FIG. 12 illustrates example method(s) 1200 of a multi-screen hold anddrag gesture. The order in which the method is described is not intendedto be construed as a limitation, and any number of the described methodblocks can be combined in any order to implement the method, or analternate method.

At block 1202, a hold input is recognized at a first screen of amulti-screen system, the hold input being recognized when held in place.For example, the input recognition system 110 recognizes the hold input1114 at the first screen 1104, where the hold input is recognized whenheld in place. At block 1204, a motion input is recognized at a secondscreen of the multi-screen system, the motion input being recognized toselect a displayed object while the hold input remains in place. Forexample, the input recognition system 110 also recognizes the motioninput 1116 at the second screen 1106, where the motion input isrecognized to select the second journal page 1112 while the hold inputremains in place.

At block 1206, a hold and drag gesture is determined from the recognizedhold and motion inputs. For example, the gesture module 108 determinesthe multi-screen hold and drag gesture from the recognized hold andmotion inputs 1114, 1116. The hold and drag gesture can be identified asa cross-screen combination of the hold and motion inputs. In variousembodiments, the hold and drag gesture is effective to maintain adisplay of a first part of the displayed object on the first screen anddrag a second part of the displayed object that is displayed on thesecond screen to pocket the second part of the displayed object for asplit-screen view (at block 1208); maintain a display of a first part ofthe displayed object on the first screen and drag a pocketed second partof the displayed object to expand the display on the second screen (atblock 1210); maintain a display of the displayed object on the firstscreen and expand the display of the displayed object onto the secondscreen (at block 1212); and/or initiate a multi-screen display of thedisplayed object (at block 1214).

Multi-Screen Hold and Page-Flip Gesture

FIG. 13 illustrates examples 1300 of a multi-screen hold and page-flipgesture on a multi-screen system 1302, which in these examples, is shownas a two-screen device. The multi-screen system 1302 may be implementedas any of the various devices described with reference to FIGS. 1 and 2.In this example, the multi-screen system 1302 includes a first screen1304 and a second screen 1306, each implemented to display any type ofuser interface and various displayable objects (e.g., any type ofpictures, images, graphics, text, notes, sketches, drawings, selectablecontrols, user interface elements, etc.). The screens can also displayjournal pages, such as any type of notebook, periodical, book, paper,single page, and the like in an electronic form. The multi-screen system1302 can include a gesture module 108 and an input recognition system110, as described with reference to the computing device 102 shown inFIG. 1, and may also be implemented with any combination of componentsas described with reference to the example device shown in FIG. 21.Although the examples are illustrated and described with reference tothe two-screen device, embodiments of a multi-screen hold and page-flipgesture can be implemented by a multi-screen system having more than twoscreens.

A multi-screen hold and page-flip gesture can be used to select ajournal page that is displayed on one screen and flip journal pages todisplay two additional or new journal pages, much like flipping pages ina book. The journal pages are flipped in a direction of the selectedjournal page to display the two new journal pages, much like flippingpages forward or backward in a book. Alternatively, a hold and page-flipgesture can be used to maintain the display of a journal page that isdisplayed on one screen and flip journal pages to display a differentjournal page on another screen. Non-consecutive journal pages can thenbe displayed side-by-side, which for a book, would involve tearing apage out of the book to place it in a non-consecutive page order to viewit side-by-side with another page. In an embodiment, a multi-screen holdand page-flip gesture is configurable to either flip journal pages todisplay two new journal pages, or maintain the display of a firstjournal page and flip the journal pages to display a different,non-consecutive second journal page side-by-side with the first journalpage.

In the first view 1308 of the multi-screen system 1302, a first journalpage 1310 is displayed on the first screen 1304, and a second journalpage 1312 is displayed on the second screen 1306. The input recognitionsystem 110 is implemented to recognize a hold input 1314 at the firstscreen 1304, where the hold input is recognized when held to select thejournal page 1310 that is displayed on the first screen 1304. The inputrecognition system 110 can also recognize a motion input 1316 at thesecond screen 1306, where the motion input is recognized while the holdinput remains in place.

The gesture module 108 is implemented to determine the multi-screen holdand page-flip gesture from the recognized hold and motion inputs 1314,1316. The hold and page-flip gesture can be identified as a cross-screencombination of the hold and motion inputs, which in various embodimentsmay include: hold and drag inputs on opposite screens with one or twoinput devices (e.g., one finger, or two fingers); and/or a hold inputand a drag input across the bezel onto the opposite screen. The hold andpage-flip gesture is effective to select the journal page 1310 on thefirst screen 1304 while one or more additional journal pages are flippedfor display. The second view 1318 of the multi-screen system 1302illustrates that two additional journal pages 1320, 1322 have beenpage-flipped for display on the respective first and second screens1304, 1306. Alternatively, the third view 1324 of the multi-screensystem 1302 illustrates that the display of journal page 1310 ismaintained on the first screen 1304 and a non-consecutive journal page1322 has been page-flipped for display side-by-side on the second screen1306.

It should be noted that the representations of the hold and motioninputs are merely illustrative for discussion purposes and may or maynot appear on the screens of the multi-screen system when describedembodiments are implemented. Additionally, any description herein of aninput or motion at one screen that may correlate to another input ormotion at another screen is applicable to either the first or secondscreens of the multi-screen system.

FIG. 14 illustrates example method(s) 1400 of a multi-screen hold andpage-flip gesture. The order in which the method is described is notintended to be construed as a limitation, and any number of thedescribed method blocks can be combined in any order to implement themethod, or an alternate method.

At block 1402, a hold input is recognized at a first screen of amulti-screen system, the hold input being recognized when held to selecta journal page that is displayed on the first screen. For example, theinput recognition system 110 recognizes the hold input 1314 at the firstscreen 1304, and the hold input is recognized when held to select thejournal page 1310 that is displayed on the first screen 1304. At block1404, a motion input is recognized at a second screen of themulti-screen system, the motion input being recognized while the holdinput remains in place. For example, the input recognition system 110also recognizes the motion input 1316 at the second screen 1306, and themotion input is recognized while the hold input remains in place.

At block 1406, a hold and page-flip gesture is determined from therecognized hold and motion inputs, and effective to select the journalpage while additional journal pages are flipped for display. Forexample, the gesture module 108 determines the multi-screen hold andpage-flip gesture from the recognized hold and motion inputs 1314, 1316.The hold and page-flip gesture is identified as a cross-screencombination of the hold and motion inputs. In various embodiments, thehold and page-flip gesture is effective to select the journal page thatis displayed on the first screen and flip journal pages (optionally in adirection of the selected journal page) to display two additionaljournal pages, one each displayed on the first and second screens (atblock 1408); maintain the display of the journal page that is displayedon the first screen and flip journal pages to display a differentjournal page on the second screen (at block 1410); and/or maintain thedisplay of the journal page that is displayed on the first screen andflip journal pages to display a non-consecutive journal page on thesecond screen side-by-side with first journal page (at block 1412).

In an embodiment, a hold and page-flip gesture is configurable to selectthe journal page that is displayed on the first screen and flip journalpages to display two additional journal pages, one each displayed on thefirst and second screens (as described with reference to block 1408), orto maintain the display of the journal page that is displayed on thefirst screen and flip the journal pages to display a different journalpage on the second screen (as described with reference to blocks 1410and 1412).

Multi-Screen Bookmark Hold Gesture

FIG. 15 illustrates examples 1500 of a multi-screen bookmark holdgesture on a multi-screen system 1502, which in these examples, is shownas a two-screen device. The multi-screen system 1502 may be implementedas any of the various devices described with reference to FIGS. 1 and 2.In this example, the multi-screen system 1502 includes a first screen1504 and a second screen 1506, each implemented to display any type ofuser interface and various displayable objects (e.g., any type ofpictures, images, graphics, text, notes, sketches, drawings, selectablecontrols, user interface elements, etc.). The screens can also displayjournal pages, such as any type of notebook, periodical, book, paper,single page, and the like in an electronic form. The multi-screen system1502 can include a gesture module 108 and an input recognition system110, as described with reference to the computing device 102 shown inFIG. 1, and may also be implemented with any combination of componentsas described with reference to the example device shown in FIG. 21.Although the examples are illustrated and described with reference tothe two-screen device, embodiments of a multi-screen bookmark holdgesture can be implemented by a multi-screen system having more than twoscreens.

A multi-screen bookmark hold gesture can be used to bookmark a journalpage at a location of a hold input to the journal page on a screen, andadditional journal pages can be flipped for viewing while the bookmarkis maintained for the journal page. A bookmark hold gesture mimics theaction of a reader holding a thumb or finger between pages to save aplace in a book while flipping through other pages of the book.Additionally, a bookmark is a selectable link back to the journal page,and a selection input of the bookmark flips back to display the journalpage on the screen.

In the first view 1508 of the multi-screen system 1502, a first journalpage 1510 is displayed on the first screen 1504, and a second journalpage 1512 is displayed on the second screen 1506. The first journal page1510 is displayed over a journal page 1514 that is bookmarked. The inputrecognition system 110 is implemented to recognize a hold input 1516 atthe first screen 1504, where the hold input is recognized when held inplace proximate an edge of the journal page 1514 that is bookmarked onthe first screen 1504. The input recognition system 110 can alsorecognize a motion input 1518 at the second screen 1506, where themotion input is recognized while the hold input remains in place. In anembodiment, the motion input 1518 is recognized at the second screen1506 along an outer edge of the journal page 1512, and the motion inputis effective to flip journal pages at 1520 while the bookmark ismaintained for the journal page 1514 on the first screen 1504.

The gesture module 108 is implemented to determine the multi-screenbookmark hold gesture from the recognized hold and motion inputs 1516,1518. The bookmark hold gesture can be identified as a cross-screencombination of the hold and motion inputs, and the gesture is effectiveto bookmark journal page 1514 at a location of the hold input 1516 onthe first screen 1504. In embodiments, a bookmark identifier 1522 isdisplayed to identify the journal page 1514 that is bookmarked and thelocation of the bookmark on the first screen. In this example, thebookmark identifier 1522 is a partial display of the journal page 1514that is bookmarked. A bookmark and/or bookmark identifier is aselectable link to the journal page 1514 that is bookmarked on the firstscreen 1504, and the input recognition system 110 can recognize aselection input of the bookmark effective to flip back and display thejournal page 1514 on the first screen.

The second view 1524 of the multi-screen system 1502 illustrates analternate hold input 1526, such as when a user may hold a two-screendevice with one hand while also bookmarking the journal page 1510 on thefirst screen 1504. The input recognition system 110 is implemented torecognize the hold input 1526 at the first screen 1504, and alsorecognize a motion input 1528 at the second screen 1506, where themotion input is recognized while the hold input remains in place. In anembodiment, the motion input 1528 is recognized at the second screen1506 and is effective to flip the journal pages while the bookmark ismaintained. In an implementation, the input recognition system 110 canrecognize a bookmark hold gesture in defined regions, such as a regionwhere a user is likely to both hold the device and bookmark a page.Alternatively or in addition, the multi-screen system 1502 may beimplemented to sense the orientation of the screens, such that pagebookmarking automatically adapts to the manner in which a user holds thedevice.

The third view 1530 of the multi-screen system 1502 illustrates that ahold input from which a bookmark is determined can include a slidemotion input 1532 proximate a corner of the journal page 1514. The slidemotion input 1532 can be recognized as a progression of motion toinitiate the hold input, and the slide motion input can be determined tobookmark the journal page 1514 at the corner. The bookmark is maintainedon the first screen 1504 for the journal page 1514 while other journalpages are flipped for viewing at 1534. In embodiments, there are varioustechniques that can be implemented to distinguish between: holding apage to temporarily save a position; explicitly “dog-earing” a page witha bookmark; or flipping back to a page denoted by a temporary hold orbookmark. In an embodiment, a hold input can be recognized to implicitlytemporarily save a page position. Then a user can simply lift the inputto discard the temporary bookmark or alternatively, provide the slidemotion input to flip back to a saved page position. In anotherembodiment, if the slide motion input is initiated at approximately asame time as the hold input, then the page dog-ear bookmark can becreated. In another embodiment, a dog-ear bookmark may only berecognized at defined locations around the boundary of a journal page(e.g., at the corners of the page), whereas the implicit temporary pagehold can be implemented for a larger area or region.

It should be noted that the representations of the hold and motioninputs are merely illustrative for discussion purposes and may or maynot appear on the screens of the multi-screen system when describedembodiments are implemented. Additionally, any description herein of aninput or motion at one screen that may correlate to another input ormotion at another screen is applicable to either the first or secondscreens of the multi-screen system.

FIG. 16 illustrates example method(s) 1600 of a multi-screen bookmarkhold gesture. The order in which the method is described is not intendedto be construed as a limitation, and any number of the described methodblocks can be combined in any order to implement the method, or analternate method.

At block 1602, a hold input is recognized at a first screen of amulti-screen system, the hold input being recognized when held in placeproximate an edge of a journal page that is displayed on the firstscreen. For example, the input recognition system 110 recognizes thehold input 1516 at the first screen 1504, and the hold input isrecognized when held in place proximate an edge of the journal page 1514that is bookmarked on the first screen 1504. The hold input can includethe slide motion input 1532 proximate a corner of the journal page 1514.The input recognition system 110 recognizes the slide motion input as aprogression of motion to initiate the hold input, and the gesture module108 determines the bookmark hold gesture from the slide motion input tobookmark the journal page.

At block 1604, a motion input is recognized at a second screen of themulti-screen system, the motion input being recognized while the holdinput remains in place. For example, input recognition system 110 alsorecognizes the motion input 1518 at the second screen 1506, and themotion input is recognized while the hold input remains in place. Theinput recognition system 110 can recognize the motion input along anouter edge of an opposite journal page that is displayed on the secondscreen 1506, and the motion input is effective to flip journal pageswhile the bookmark is maintained for the journal page 1514 on the firstscreen 1504.

At block 1606, a bookmark hold gesture is determined from the recognizedhold and motion inputs, the bookmark hold gesture effective to bookmarkthe journal page at a location of the hold input on the first screen.For example, the gesture module 108 determines the multi-screen bookmarkhold gesture from the recognized hold and motion inputs 1516, 1518. Thebookmark hold gesture can be identified as a cross-screen combination ofthe hold and motion inputs. A bookmark and/or bookmark identifier is aselectable link to a journal page that is bookmarked on the first screen1504, and the input recognition system 110 recognizes a selection inputof the bookmark effective to flip back and display the journal page onthe first screen.

At block 1608, display a bookmark identifier to identify the journalpage that is bookmarked and the location of the bookmark on the firstscreen. For example, a bookmark identifier 1522 is displayed to identifythe journal page 1514 that is bookmarked and the location of thebookmark on the first screen. In an implementation, the bookmarkidentifier 1522 can be a partial display of the journal page itself thatis bookmarked.

Multi-Screen Object-Hold and Page-Change Gesture

FIG. 17 illustrates examples 1700 of a multi-screen object-hold andpage-change gesture on a multi-screen system 1702, which in theseexamples, is shown as a two-screen device. The multi-screen system 1702may be implemented as any of the various devices described withreference to FIGS. 1 and 2. In this example, the multi-screen system1702 includes a first screen 1704 and a second screen 1706, eachimplemented to display any type of user interface and variousdisplayable objects (e.g., any type of pictures, images, graphics, text,notes, sketches, drawings, selectable controls, user interface elements,etc.). The screens can also display journal pages, such as any type ofnotebook, periodical, book, paper, single page, and the like in anelectronic form. The multi-screen system 1702 can include a gesturemodule 108 and an input recognition system 110, as described withreference to the computing device 102 shown in FIG. 1, and may also beimplemented with any combination of components as described withreference to the example device shown in FIG. 21. Although the examplesare illustrated and described with reference to the two-screen device,embodiments of a multi-screen object-hold and page-change gesture can beimplemented by a multi-screen system having more than two screens.

A multi-screen object-hold and page-change gesture can be used to moveand/or copy a displayed object (or objects) from one display location toanother, such as to incorporate a displayed object for display on ajournal page. Additionally, a relative display position can bemaintained when a displayed object is moved or copied from one displaylocation to another. This can also include a selection of multipleobjects that are selected with tap-selection inputs to a series ofobjects in succession, and then a hold input holds the selection while amotion input is recognized to change journal pages. This gesture canthen be determined to move and/or copy all of the held objects to a newjournal page that is displayed, while maintaining their relative displayposition and/or the relative spatial relationship between the objects.Alternatively or in addition, this gesture can include selections ofobjects that start on one page, then are held while journal pages areflipped, and additional objects from other pages are selected to beadded to the selection of objects and carried along with the group.

In the first view 1708 of the multi-screen system 1702, a first journalpage 1710 is displayed on the first screen 1704, and a second journalpage 1712 is displayed on the second screen 1706. The input recognitionsystem 110 is implemented to recognize a hold input 1714 at the firstscreen 1704, where the hold input is recognized when held to select adisplayed object 1716 on the first screen 1704. The input recognitionsystem 110 can also recognize a motion input 1718 at the second screen1706, where the motion input is recognized while the displayed object1716 is selected and the motion input effective to change journal pagesat 1720. When the journal pages are changed at 1720, a subsequentjournal page 1722 is revealed for display. In an embodiment, the motioninput 1518 is recognized at the second screen 1506 along an outer edgeof the journal page 1512, and the motion input is effective to flipjournal pages at 1520 while the bookmark is maintained for the journalpage 1514 on the first screen 1504.

The gesture module 108 is implemented to determine the multi-screenobject-hold and page-change gesture from the recognized hold and motioninputs 1714, 1718. The object-hold and page-change gesture can beidentified as a cross-screen combination of the hold and motion inputs,and the gesture is effective to move or copy the displayed object 1716for display on a currently displayed journal page. The second view 1724of the multi-screen system 1702 illustrates that the displayed object1716 is moved from journal page 1710 (e.g., or copied from journal page1710) for display on a currently displayed journal page 1726, which isdisplayed on the first screen 1704. The displayed object 1716 remainsselected while journal pages are changed. The input recognition system110 can then recognize that the displayed object 1716 is released fromthe hold input, and the object-hold and page-change gesture is effectiveto move or copy the displayed object for display on a currentlydisplayed journal page. Additionally, a relative display position of thedisplayed object can be maintained when the displayed object is moved orcopied from one display location to another.

It should be noted that the representations of the hold and motioninputs are merely illustrative for discussion purposes and may or maynot appear on the screens of the multi-screen system when describedembodiments are implemented. Additionally, any description herein of aninput or motion at one screen that may correlate to another input ormotion at another screen is applicable to either the first or secondscreens of the multi-screen system.

FIG. 18 illustrates example method(s) 1800 of a multi-screen object-holdand page-change gesture. The order in which the method is described isnot intended to be construed as a limitation, and any number of thedescribed method blocks can be combined in any order to implement themethod, or an alternate method.

At block 1802, a hold input is recognized at a first screen of amulti-screen system, the hold input being recognized when held to selecta displayed object on the first screen. For example, the inputrecognition system 110 recognizes the hold input 1714 at the firstscreen 1704, and the hold input is recognized when held to select thedisplayed object 1716 on the first screen 1704. At block 1804, a motioninput is recognized at a second screen of the multi-screen system, themotion input being recognized while the displayed object is selected andthe motion input effective to change one or more journal pages. Forexample, the input recognition system 110 also recognizes the motioninput 1718 at the second screen 1706, the motion input being recognizedwhile the displayed object 1716 is selected and the motion inputeffective to change journal pages at 1720.

At block 1806, an object-hold and page-change gesture is determined fromthe recognized hold and motion inputs. For example, the gesture module108 determines the multi-screen object-hold and page-change gesture fromthe recognized hold and motion inputs 1714, 1718. The object-hold andpage-change gesture can be identified as a cross-screen combination ofthe hold and motion inputs. In an embodiment, the object-hold andpage-change gesture is effective to initiate a copy and paste functionto copy the displayed object 1716 for display on a currently displayedjournal page 1726.

At block 1808, the displayed object is recognized when released from thehold input, and the object-hold and page-change gesture effective tomove and/or copy the displayed object for display on a currentlydisplayed journal page. For example, the input recognition system 110recognizes when the displayed object 1716 is released from the holdinput, and the object-hold and page-change gesture is effective to moveor copy the displayed object for display on a currently displayedjournal page. The second view 1724 of the multi-screen system 1702illustrates that the displayed object 1716 is moved from journal page1710 (e.g., or copied from journal page 1710) for display on a currentlydisplayed journal page 1726, which is displayed on the first screen1704. Additionally, a relative display position of the displayed objectis maintained when the displayed object is moved or copied from onedisplay location to another. An object-hold and page-change gesture mayalso be effective to select multiple displayed objects that are movedand/or copied as a group from one display location to another.

Multi-Screen Synchronous Slide Gesture

FIG. 19 illustrates examples 1900 of a multi-screen synchronous slidegesture on a multi-screen system 1902, which in these examples, is shownas a two-screen device. The multi-screen system 1902 may be implementedas any of the various devices described with reference to FIGS. 1 and 2.In this example, the multi-screen system 1902 includes a first screen1904 and a second screen 1906, each implemented to display any type ofuser interface and various displayable objects (e.g., any type ofpictures, images, graphics, text, notes, sketches, drawings, selectablecontrols, user interface elements, etc.). The screens can also displayjournal pages, such as any type of notebook, periodical, book, paper,single page, and the like in an electronic form. The multi-screen system1902 can include a gesture module 108 and an input recognition system110, as described with reference to the computing device 102 shown inFIG. 1, and may also be implemented with any combination of componentsas described with reference to the example device shown in FIG. 21.Although the examples are illustrated and described with reference tothe two-screen device, embodiments of a multi-screen synchronous slidegesture can be implemented by a multi-screen system having more than twoscreens.

A multi-screen synchronous slide gesture can be used to move a displayedobject from one screen for display on another screen, replace displayedobjects on the system screens with different displayed objects, movedisplayed objects to reveal a workspace on the system screens, and/orcycle through one or more workspaces (e.g., applications, interfaces,etc.) that are displayed on the system screens. A synchronous slidegesture may also be used to navigate to additional views, or reassign acurrent view to a different screen. Additionally, different applicationsor workspaces can be kept on a stack and cycled through, forward andback, with synchronous slide gestures.

In the first view 1908 of the multi-screen system 1902, a journal page1910 is displayed as being moved from the first screen 1904 for displayon the second screen 1906. The input recognition system 110 isimplemented to recognize a first motion input 1912 at the first screen1904 when the first motion input moves in a particular direction acrossthe first screen. The input recognition system 110 can also recognize asecond motion input 1914 at the second screen 1906 when the secondmotion input moves in the particular direction across the second screenand approximately when the first motion input is recognized.

The gesture module 108 is implemented to determine the multi-screensynchronous slide gesture from the recognized motion inputs 1912, 1914.The synchronous slide gesture can be identified as a cross-screencombination of the motion inputs, and the gesture is effective to movethe journal page 1910 from display on the first screen 1904 to displayon the second screen 1906.

In the second view 1916 of the multi-screen system 1902, the firstjournal page 1910 that is displayed on the first screen 1904 and asecond journal page 1918 that is displayed on the second screen 1906 areillustrated as being replaced with different journal pages. The inputrecognition system 110 can recognize a first motion input 1920 at thefirst screen 1904 when the first motion input moves in a particulardirection across the first screen. The input recognition system 110 canalso recognize a second motion input 1922 at the second screen 1906 whenthe second motion input moves in the particular direction across thesecond screen and approximately when the first motion input isrecognized. The gesture module 108 can determine the multi-screensynchronous slide gesture from the recognized motion inputs 1920, 1922.As shown in the third view 1924 of the multi-screen system 1902, thesynchronous slide gesture is effective to move and/or replace thejournal pages 1910, 1918 with different journal pages 1926, 1928 fordisplay on the system screens.

It should be noted that the various representations of the motion inputsare merely illustrative for discussion purposes and may or may notappear on the screens of the multi-screen system when describedembodiments are implemented. Additionally, any description herein of aninput or motion at one screen that may correlate to another input ormotion at another screen is applicable to either the first or secondscreens of the multi-screen system.

FIG. 20 illustrates example method(s) 2000 of a multi-screen synchronousslide gesture. The order in which the method is described is notintended to be construed as a limitation, and any number of thedescribed method blocks can be combined in any order to implement themethod, or an alternate method.

At block 2002, a first motion input is recognized at a first screen of amulti-screen system when moving in a particular direction across thefirst screen. For example, the input recognition system 110 recognizesthe first motion input 1912 at the first screen 1904 when the firstmotion input moves in a particular direction across the first screen. Atblock 2004, a second motion input is recognized at a second screen ofthe multi-screen system when moving in the particular direction acrossthe second screen and approximately when the first motion input isrecognized. For example, the input recognition system 110 alsorecognizes the second motion input 1914 at the second screen 1906 whenthe second motion input moves in the particular direction across thesecond screen and approximately when the first motion input isrecognized.

At block 2006, a synchronous slide gesture is determined from therecognized first and second motion inputs. For example, the gesturemodule 108 determines the multi-screen synchronous slide gesture fromthe recognized motion inputs 1912, 1914. The synchronous slide gestureis identified as a cross-screen combination of the first and secondmotion inputs. In various embodiments, the synchronous slide gesture iseffective to move a displayed object from display on the first screen todisplay on the second screen (at block 2008); replace one or moredisplayed objects on the first and second screens with differentdisplayed objects (at block 2010); move one or more displayed objectsand reveal a workspace on the first and second screens (at block 2012);cycle through one or more workspaces that are displayed on the first andsecond screens (at block 2014); and/or replace one or more applicationson the first and second screens with different applications (at block2016).

FIG. 21 illustrates various components of an example device 2100 thatcan be implemented as any type of portable and/or computing device asdescribed with reference to FIGS. 1 and 2 to implement embodiments ofmulti-screen gestures. In embodiments, device 2100 can be implemented asany one or combination of a wired and/or wireless device, a multi-screendevice, as any form of television client device (e.g., televisionset-top box, digital video recorder (DVR), etc.), consumer device,computer device, server device, portable computer device, user device,communication device, video processing and/or rendering device,appliance device, gaming device, electronic device, and/or as any othertype of device. Device 2100 may also be associated with a user (i.e., aperson) and/or an entity that operates the device such that a devicedescribes logical devices that include users, software, firmware, and/ora combination of devices.

Device 2100 includes communication devices 2102 that enable wired and/orwireless communication of device data 2104 (e.g., received data, datathat is being received, data scheduled for broadcast, data packets ofthe data, etc.). The device data 2104 or other device content caninclude configuration settings of the device, media content stored onthe device, and/or information associated with a user of the device.Media content stored on device 2100 can include any type of audio,video, and/or image data. Device 2100 includes one or more data inputs2106 via which any type of data, media content, and/or inputs can bereceived, such as user-selectable inputs, messages, music, televisionmedia content, recorded video content, and any other type of audio,video, and/or image data received from any content and/or data source.

Device 2100 also includes communication interfaces 2108 that can beimplemented as any one or more of a serial and/or parallel interface, awireless interface, any type of network interface, a modem, and as anyother type of communication interface. The communication interfaces 2108provide a connection and/or communication links between device 2100 anda communication network by which other electronic, computing, andcommunication devices communicate data with device 2100.

Device 2100 includes one or more processors 2110 (e.g., any ofmicroprocessors, controllers, and the like) which process variouscomputer-executable instructions to control the operation of device 2100and to implement embodiments of multi-screen gestures. Alternatively orin addition, device 2100 can be implemented with any one or combinationof hardware, firmware, or fixed logic circuitry that is implemented inconnection with processing and control circuits which are generallyidentified at 2112. Although not shown, device 2100 can include a systembus or data transfer system that couples the various components withinthe device. A system bus can include any one or combination of differentbus structures, such as a memory bus or memory controller, a peripheralbus, a universal serial bus, and/or a processor or local bus thatutilizes any of a variety of bus architectures.

Device 2100 also includes computer-readable media 2114, such as one ormore memory components, examples of which include random access memory(RAM), non-volatile memory (e.g., any one or more of a read-only memory(ROM), flash memory, EPROM, EEPROM, etc.), and a disk storage device. Adisk storage device may be implemented as any type of magnetic oroptical storage device, such as a hard disk drive, a recordable and/orrewriteable compact disc (CD), any type of a digital versatile disc(DVD), and the like. Device 2100 can also include a mass storage mediadevice 2116.

Computer-readable media 2114 provides data storage mechanisms to storethe device data 2104, as well as various device applications 2118 andany other types of information and/or data related to operationalaspects of device 2100. For example, an operating system 2120 can bemaintained as a computer application with the computer-readable media2114 and executed on processors 2110. The device applications 2118 caninclude a device manager (e.g., a control application, softwareapplication, signal processing and control module, code that is nativeto a particular device, a hardware abstraction layer for a particulardevice, etc.).

The device applications 2118 also include any system components ormodules to implement embodiments of multi-screen gestures. In thisexample, the device applications 2118 can include interface applications2122 and a gesture module 2124, such as when device 2100 is implementedas a multi-screen device. The interface applications 2122 and thegesture module 2124 are shown as software modules and/or computerapplications. Alternatively or in addition, the interface applications2122 and/or the gesture module 2124 can be implemented as hardware,software, firmware, or any combination thereof.

Device 2100 includes an input recognition system 2126 implemented torecognize various inputs or combinations of inputs, such as a selectinput, hold input, motion input, touch input, tap input, and the like.The input recognition system 2126 may include any type of inputdetection features to distinguish the various types of inputs, such assensors, light sensing pixels, touch sensors, cameras, and/or a naturaluser interface that interprets user interactions, gestures, inputs, andmotions.

Device 2100 also includes an audio and/or video rendering system 2128that generates and provides audio data to an audio system 2130 and/orgenerates and provides display data to a display system 2132. The audiosystem 2130 and/or the display system 2132 can include any devices thatprocess, display, and/or otherwise render audio, display, and imagedata. Display data and audio signals can be communicated from device2100 to an audio device and/or to a display device via an RF (radiofrequency) link, S-video link, composite video link, component videolink, DVI (digital video interface), analog audio connection, or othersimilar communication link. In an embodiment, the audio system 2130and/or the display system 2132 are implemented as external components todevice 2100. Alternatively, the audio system 2130 and/or the displaysystem 2132 are implemented as integrated components of example device2100.

Although embodiments of multi-screen gestures have been described inlanguage specific to features and/or methods, it is to be understoodthat the subject of the appended claims is not necessarily limited tothe specific features or methods described. Rather, the specificfeatures and methods are disclosed as example implementations of themulti-screen gestures.

The invention claimed is:
 1. A computer-implemented method, comprising:recognizing a hold input at a first screen of a multi-screen system, thehold input being recognized when held in place; recognizing a motioninput at a second screen of the multi-screen system, the motion inputbeing recognized to select a displayed object while the hold inputremains in place; and determining a hold and drag gesture from therecognized hold and motion inputs, the hold and drag gesture effectiveto maintain a display of a first part of the displayed object on thefirst screen and drag a second part of the displayed object that isdisplayed on the second screen to condense and pocket the second part ofthe displayed object proximate a bezel between the first and secondscreens for a split-screen view.
 2. A method as recited in claim 1,wherein the hold and drag gesture is identified as a cross-screencombination of the hold and motion inputs.
 3. A method as recited inclaim 1, wherein the condensed and pocketed second part of the displayedobject is a selectable link to the second part of the displayed object.4. A method as recited in claim 1, wherein the hold and drag gesture iseffective to maintain a display of a first part of the displayed objecton the first screen and drag a condensed and pocketed second part of thedisplayed object to expand the display on the second screen.
 5. A methodas recited in claim 1, wherein the hold and drag gesture is effective tomaintain a display of the displayed object on the first screen andexpand the display of the displayed object onto the second screen.
 6. Amethod as recited in claim 1, wherein the hold and drag gesture iseffective to initiate a multi-screen display of the displayed object. 7.A method as recited in claim 1, wherein the motion input at the secondscreen is recognized to select the displayed object while the hold inputremains in contact with the first screen at a selection position of thehold input.
 8. A multi-screen system, comprising: at least a memory anda processor to implement a gesture module configured to determine a holdand drag gesture from recognized hold and motion inputs, the hold anddrag gesture effective to maintain a display of a first part of adisplayed object on a first screen and drag a condensed and pocketedsecond part of the displayed object to expand the display on a secondscreen; an input recognition system configured to: recognize the holdinput at the first screen, the hold input recognized when held in place;and recognize the motion input at the second screen, the motion inputrecognized to select the displayed object while the hold input remainsin place.
 9. A multi-screen system as recited in claim 8, wherein thegesture module is further configured to identify the hold and draggesture as a cross-screen combination of the hold and motion inputs. 10.A multi-screen system as recited in claim 8, wherein the hold and draggesture is effective to maintain a display of a first part of thedisplayed object on the first screen and drag a second part of thedisplayed object that is displayed on the second screen to condense andpocket the second part of the displayed object proximate a bezel betweenthe first and second screens for a split-screen view.
 11. A multi-screensystem as recited in claim 10, wherein the condensed and pocketed secondpart of the displayed object is a selectable link to the second part ofthe displayed object.
 12. A multi-screen system as recited in claim 8,wherein the hold and drag gesture is effective to maintain a display ofthe displayed object on the first screen and expand the display of thedisplayed object onto the second screen.
 13. A multi-screen system asrecited in claim 8, wherein the hold and drag gesture is effective toinitiate a multi-screen display of the displayed object.
 14. Amulti-screen system as recited in claim 8, wherein the input recognitionsystem is configured to recognize the motion input to select thedisplayed object at the second screen while the hold input remains incontact with the first screen at a selection position of the hold input.15. Computer-readable storage media comprising instructions that areexecutable and, responsive to executing the instructions, a multi-screensystem is initiated to: recognize a hold input at a first screen of themulti-screen system, the hold input recognized when held in place;recognize a motion input at a second screen of the multi-screen system,the motion input recognized to select a displayed object while the holdinput remains in place; determine a hold and drag gesture from therecognized hold and motion inputs; and maintain a display of a firstpart of the displayed object on the first screen and drag a second partof the displayed object that is displayed on the second screen tocondense and pocket the second part of the displayed object proximate abezel between the first and second screens for a split-screen view. 16.Computer-readable storage media as recited in claim 15, furthercomprising additional instructions that are executable and, responsiveto executing the additional instructions, the multi-screen system isfurther initiated to identify the hold and drag gesture as across-screen combination of the hold and motion inputs. 17.Computer-readable storage media as recited in claim 15, furthercomprising additional instructions that are executable and, responsiveto executing the additional instructions, the multi-screen system isfurther initiated to maintain a display of a first part of the displayedobject on the first screen and drag a condensed and pocketed second partof the displayed object to expand the display on the second screen. 18.Computer-readable storage media as recited in claim 15, furthercomprising additional instructions that are executable and, responsiveto executing the additional instructions, the multi-screen system isfurther initiated to maintain a display of the displayed object on thefirst screen and expand the display of the displayed object onto thesecond screen.
 19. Computer-readable storage media as recited in claim15, further comprising additional instructions that are executable and,responsive to executing the additional instructions, the multi-screensystem is further initiated to initiate a multi-screen display of thedisplayed object in response to the hold and drag gesture. 20.Computer-readable storage media as recited in claim 15, furthercomprising additional instructions that are executable and, responsiveto executing the additional instructions, the multi-screen system isfurther initiated to recognize the motion input at the second screen toselect the displayed object while the hold input remains in contact withthe first screen at a selection position of the hold input.